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ABSTRACT 


Two  training  squadrons  of  student  naval  aviators  flying  the  T-2C  Buckeye  aircraft 
during  their  Intermediate  Stage  of  Strike  (Jet)  Training  were  compared  to  identify  any 
effects  attributed  to  exposure  to  the  T-45  Operational  Flight  Trainer,  a  state-of-the-art, 
visual,  dome  simulator.  Students  in  VT-23  at  Naval  Air  Station  (NAS)  Kingsville,  Texas 
were  exposed  to  demonstrations  of  carrier  landings  in  the  T-45  simulator  prior  to  their 
carrier  qualifications.  A  sister  squadron  at  NAS  Meridian,  Mississippi  (VT-19)  did  not 
receive  exposure  to  the  simulator.  Both  squadrons  conducted  carrier  qualifications  on  the 
same  dates  and  same  aircraft  carrier.  A  comparison  of  the  squadrons  considered  the  four 
measures  of  effectiveness  (MOE):  1)  squadron  disqualification  rates,  2)  LSO  trend 
analysis  scores,  3)  Aviation  Training  Form  (ATF)  CQ-llx  performance  evaluations,  and 
4)  average  grades  for  individual  performance  areas  listed  on  ATF  CQ-llx.  Students  in 
VT-23  had  higher  CQ-1  lx  performance  evaluations.  When  the  performance  evaluations 
are  broken  down  to  individual  performance  areas,  VT-23  also  has  substantially  higher 
grades  in  Pattern.  Allowing  for  the  importance  that  a  proper  flight  pattern  has  on  the 
approach  and  landing  on  the  aircraft  carrier,  it  is  argued  that  exposure  to  the  T-45  visual 
simulator  had  a  positive  effect  on  the  students  in  VT-23. 
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EXECUTIVE  SUMMARY 


With  the  introduction  of  a  new  training  jet,  the  T-45A  Goshawk,  the  Navy  is 
introducing  an  entirely  new  training  program.  The  T-45  training  program  is  heavily 
computerized,  emphasizing  computer  based  training  and  a  state-of-the-art,  visual,  dome 
simulator.  When  Hughes  designed  the  new  training  jet,  they  designed  more  than  just  an 
aircraft,  they  designed  an  entire  training  system,  including  classroom  lectures,  computer 
based  training,  and  two  highly  advanced,  state-of-the-art  simulators.  The  most  advanced 
of  these  flight  simulators  introduces  visual  simulation  into  the  training  pipeline. 

The  T-45  Operational  Flight  Training  (OFT)  simulator  is  built  around  a  replica  of 
a  Goshawk  cockpit.  The  visual  simulation  is  projected  onto  a  partial  dome  screen, 
allowing  the  student  a  1 80  degree  horizontal  and  60  degree  vertical  field  of  view.  The 
three  areas  in  the  training  curriculum  that  require  the  greatest  use  of  the  Student  Naval 
Aviator’s  (SNA)  visual  skills  are  landing  skills,  weapons  training  and  carrier 
qualifications.  The  visual  system  allows  extensive  training  in  all  of  these  areas.  For 
weapons  training,  the  simulator  includes  air/ground  bomb,  racket  and  strafing  models, 
rocket  models,  and  the  dust  puffs  of  ordnance  exploding  on  the  ground  at  the  impact 
points.  For  landing  skills,  several  highly  detailed  models  of  Naval  Air  Stations,  outlying 
fields  and  Air  Force  Bases  are  also  modeled.  Finally,  for  carrier  operations,  a  dynamic 
model  of  the  aircraft  carrier  USS  Nimitz  is  provided.  In  the  simulator,  the  SNA  can 
experience  carrier  deck  arrested  landings,  on  deck  taxiing,  and  catapult  takeoffs  in  sea 
states  from  zero  to  sea  state  five.  Rain,  fog,  cloud  cover,  sea  state,  and  turbulence  can 
be  introduced,  modified,  or  dismissed  by  simple  instructor-computer  interface  controls  in 
order  to  provide  a  more  realistic  aircraft  flight  deck  environment. 

Such  innovation  in  training,  not  to  mention  the  cost  of  such  innovation,  begs  the 
question  of  effectiveness.  How  effective  is  the  training  program?  How  effective  is  the 
visual  simulator  over  the  standard  instrument  simulators?  The  U.S.  Navy  Jet 
Undergraduate  Training  Program  (JUTP)  is  conducted  in  two  locations.  Meridian, 
Mississippi  (VT-19)  and  Kingsville,  Texas  (VT-23).  The  T-45  visual-dome  simulator  was 
installed  in  Kingsville  in  1990.  From  1990  to  1993  the  T-45  Goshawk  was  not  ready  for 
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students.  Instructor  training  for  the  T-45  began  in  the  summer  of  1993  and  students 
started  actual  T-45  flights  nearly  a  year  later.  However,  while  the  simulator  waited  in 
Kingsville,  T-2C  Buckeye  students  were  exposed  to  demonstrations  in  the  flight  simulator 
prior  to  their  carrier  qualifications  and  weapons  training.  The  T-2C  students  in  Meridian 
did  not  receive  visual  simulator  exposure.  By  comparing  these  two  groups  in  Meridian 
and  Kingsville  it  may  be  possible  to  quantify  and  hopefully  analyze  the  effectiveness  of 
the  new  visual  simulator. 

Sixty-nine  students  from  VT-19  and  67  students  from  VT-23  were  reviewed  for 
this  study.  Both  squadrons  conducted  carrier  qualifications  on  the  same  dates  and  same 
aircraft  carrier  during  May  and  July  of  1993.  VT-19  used  five  different  Landing  Signal 
Officers  (LSO)  over  this  period.  VT-23  used  seven  different  LSO's.  Measures  of 
effectiveness  (MOE)  for  the  comparison  were  1)  squadron  disqualification  rates,  2)  LSO 
trend  analysis  scores,  3)  Aviation  Training  Form  (ATF)  CQ-llx  average  performance 
evaluations,  and  4)  average  grades  for  individual  performance  areas  listed  on  ATF  CQ- 
llx.  The  MOE’s  were  evaluated  using  a  regression  model  that  tested  for  squadron, 
month,  LSO,  and  interaction  effects. 

The  resulting  analyses  indicate  that  there  was  no  difference  between  the  squadrons 
in  disqualification  rates  and  LSO  trend  analysis  scores.  A  significant  difference  exists  in 
the  CQ-1  lx  average  performance  evaluations.  VT-23  appears  to  have  a  higher  average, 
albeit  over  a  very  small  range.  When  the  average  performance  evaluations  are  broken 
down  to  individual  performance  areas,  the  difference  between  the  squadrons  is  more 
acute.  VT-23  has  substantially  higher  grades  in  Pattern,  and  moderately  higher  grades  in 
Start  Position,  Power  Control,  and  Glideslope  Control.  Ruling  out  possible  effects  due 
to  differences  in  the  LSO's,  the  Pattern  grade  remains  as  the  most  significant  difference 
between  the  squadrons. 

While  it  must  be  argued  that  the  measures  of  effectiveness  considered  are 
subjective  measures  of  human  evaluation,  the  importance  that  a  proper  pattern  has  on  the 
entire  carrier  landing  process  cannot  be  argued.  Although  students  are  taught  to  make 
most  of  their  turns  during  the  landing  pattern  using  the  cockpit  instruments,  they  are  also 
encouraged  to  use  visual  landmarks  on  the  ground  while  learning  the  approach.  These 
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visual  aids  are  not  available  to  the  student  during  a  carrier  approach,  and  while  the 
student  is  constantly  reminded  of  this  fact,  it  must  certainly  be  disorientating  during  the 
first  few  approaches  to  the  ship.  One  advantage  of  the  T-45  visual  simulator,  may  be  an 
awareness  of  this  spatial  disorientation.  In  the  simulator,  the  student  is  given  an 
opportunity  to  observe  the  relatively  small  deck  of  a  carrier  against  a  massive,  featureless 
ocean,  and  has  the  opportunity  to  witness  a  simulated  instrument  approach  to  the  carrier. 
Such  awareness  may  encourage  students  to  place  more  emphasis  on  their  pattern,  thus 
resulting  in  higher  grades. 

Due  to  the  limited  nature  of  this  study,  it  is  imprudent  to  make  a  blanket  statement 
about  the  effectiveness  of  the  T-45  visual  simulator.  However,  although  further  study  is 
warranted,  the  simulator  appears  to  have  a  positive  effect  on  student's  ability  to  recognize 
and  fly  the  proper  pattern  as  they  approach  the  aircraft  carrier. 
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I.  INTRODUCTION 


A.  A  BRIEF  HISTORY  OF  THE  USE  OF  SIMULATORS  IN  AVIATION 

Since  the  inception  of  flight,  the  aviation  community  has  looked  for  better,  safer 
methods  to  train  new  pilots  with  minimal  loss  of  students  and  aircraft.  Flight  simulators 
allow  an  instructor  to  present  many  of  the  skills  required  to  control  an  aircraft  while  the 
student  is  still  on  the  ground.  Simulator  use  is  recorded  as  early  as  1910  in  the  United 
States,  Britain,  and  Italy,  an  indication  of  the  importance  of  this  new  form  of  training. 
Early  simulators  were  actual  modified  aircraft  mounted  on  universal  joints  that,  under 
proper  wind  conditions,  allowed  the  student  to  control  for  air  turbulence.  Other  inventors 
engineered  manual  levers  for  the  instructor  to  manipulate  the  simulator’s  movements.  The 
common  technique  stressed  by  these  early  simulators  was  movement— the  students  ability 
to  manually  control  the  aircraft. 

In  1929,  Edwin  Link  designed  a  pneumatically  controlled  simulator.  Link's 
simulator  still  required  pitch,  roll,  and  yaw  movements  to  be  initiated  manually,  but  then 
used  a  pneumatic  bellows  to  fine  tune  the  movement.  The  Link  simulator  was  announced 
to  have  simulated  the  true  feel  of  flying  without  ever  leaving  the  ground.  (Rolfe  and 
Staples  1986,  19-20).  But  just  as  Link's  patent  cleared,  flight  simulation  began  to  take 
another  direction.  Instructors  began  to  realize  that  simulator  motion  was  not  an  effective 
training  aid.  As  aircraft  became  more  involved,  instruments  were  installed  on  the 
simulators  to  mimic  the  flight  controls  on  the  more  modem  aircraft.  Simulators  proved 
to  be  excellent  instrument  trainers,  an  important  task  that  they  still  perform  today. 

By  the  beginning  of  World  War  II,  military  air  corps  in  Europe,  the  United  States, 
and  Japan  were  conducting  basic  instmment  training  in  Link  simulators.  During  World 
War  II,  simulators  were  used  for  much  more  than  training  new  pilots.  Simulators  were 
modified  to  enhance  celestial  navigation  and  bombing  skills.  World  War  II  also  saw 
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simulators  being  adapted  to  train  for  specific  types  of  aircraft.  By  the  end  of  World  War 
II,  the  aviation  world  realized  that  they  depended  more  on  the  simulator's  instrument 
capabilities  and  less  on  the  movement  of  the  simulator.  Post  World  War  II  development 
stressed  electronic  and  analog  instrument  trainers  as  the  flight  simulator's  primary  mission. 

Today,  flight  simulators  come  in  many  forms,  from  simple  static  cockpits  that  allow 
pilots  to  practice  ejection  techniques  to  the  most  advanced  F/A-1 8  trainer  that  incorporates 
motion,  pressure  suits  and  360-degree  visual,  dome  simulation.  With  the  advances  in 
computer  generated  graphics,  visual  flight  simulators  provide  a  degree  of  reality  never 
before  experienced  in  flight  simulation.  A  pilot  can  now  experience  the  feeling  of  three- 
dimensional  flight  without  ever  leaving  the  ground. 

B.  THE  SIMULATOR  AND  U.S.  NAVAL  AVIATION  TRAINING 

Throughout  the  development  of  flight  simulation,  the  military  has  been  at  the 
forefront  in  its  support  and  use  of  the  technology.  The  military  realized  the  potential  of 
the  aircraft's  fighting  capability  during  World  War  I.  At  this  time,  the  military  also 
recognized  the  need  for  rapidly  teaching  flying  skills  to  large  numbers  of  men  (Rolfe  and 
Staples  1986,  16).  While  simulators  were  not  used  extensively  during  World  War  I,  the 
need  to  optimize  training  resources  and  student  qualification  inspired  the  development  of 
the  discipline  of  Aviation  Psychology.  Aviation  Psychology  would  not  only  develop 
guidelines  for  the  military's  selection  of  recruits  for  flight  training,  but  advise  the 
engineers  who  built  the  simulators  used  in  their  training. 

During  World  War  II,  the  U.S.  Navy  trained  with  a  simulator  built  by  Bell 
Laboratories  (Rolfe  and  Staples  1986,  28).  The  Bell  Laboratories  simulator  consisted  of 
a  PBM-3  front  fuselage  and  cockpit,  with  complete  controls,  instrumentation  and  auxiliary 
equipment,  and  even  included  a  computing  device  to  solve  flight  equations.  The  PBM-3 
simulator  is  credited  as  the  first  operational  flight  trainer  that  attempted  to  simulate  the 
characteristics  of  a  particular  aircraft. 

Today  the  United  States  Navy  has  a  flight  simulator  for  every  aviation  platform  it 
flies.  In  addition  to  real  flight  hours,  pilots  are  required  to  fly  simulated  flights  on  a 
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regular  basis.  Student  pilots  begin  using  flight  simulators  in  basic  flight  school  and 
continue  into  their  specialized  training  in  either  the  Strike  (Jet),  Fixed  Wing,  or  Rotor 
communities.  In  the  flight  training  pipeline,  instrument  training  is  stressed.  Most  pilots 
do  not  fly  a  visual  simulator  until  they  complete  their  training  program  and  report  to  their 
first  squadron. 

1.  U.S.  Navy  Jet  Undergraduate  Training  Program  (JUTP) 

In  the  past,  the  training  pipeline  for  jet  pilots  consisted  of  three  distinct  stages  using 
three  different  aircraft.  The  Primary  stage  was  common  to  naval  aviators,  basic  flight 
training  in  the  twin  turbo-prop  T-34.  Once  basic  flight  training  was  completed,  SNA's 
assigned  to  the  jet  training  pipeline  began  the  Intermediate  stage  of  their  instruction.  The 
Intermediate  stage  used  the  T-2C  Buckeye  aircraft.  This  stage  lasted  for  approximately 
8  months  and  concluded  with  initial  carrier  qualification.  From  the  Intermediate  stage  the 
Student  Naval  Aviator  (SNA)  progressed  to  the  Advanced  stage  and  yet  another  aircraft, 
this  time  the  A-4  Skyhawk.  In  the  Skyhawk,  the  SNA  practiced  many  of  the  skills 
learned  previously  in  the  T-2C.  Upon  completion  of  the  Advanced  stage,  usually  about 
10  months,  the  student  qualified  for  carrier  landings  for  the  second  time.  Successful 
completion  of  all  three  stages  certified  the  SNA  as  fully  qualified,  "winged"  Navy  jet 
pilot.  The  SNA  was  assigned  to  a  specific  Navy  platform  (F-16,  A-6,  F/A-18)  and 
transferred  to  that  aircraft's  Fleet  Readiness  Squadron  (FRS).  While  training  simulators 
of  all  types  were  used  in  the  Jet  pipeline,  the  simulators  stressed  instrument  work  and 

provided  no  visual  simulation. 

2.  T-45  Goshawk  Training  Program 

With  the  introduction  of  a  new  training  jet,  the  T-45A  Goshawk,  the  Navy  is 
introducing  an  entirely  new  training  program.  The  T-45  training  program  is  heavily 
computerized,  emphasizing  computer  based  training  and  a  state-of-the-art,  visual,  dome 
simulator.  The  T-45  A  Goshawk  aircraft  was  designed  to  eliminate  the  Intermediate  stage 
of  the  jet  training  pipeline.  New  SNA's  will  now  fly  only  one  platform  after  their 
Primary  Stage.  The  T-45  curriculum  is  designed  to  last  approximately  1  year  and  covers 
all  of  the  skills  taught  in  the  original  training  pipeline,  with  one  important  difference,  only 
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one  carrier  qualification  is  now  required.  When  Hughes  designed  the  new  training  jet, 
they  designed  more  than  just  an  aircraft,  they  designed  an  entire  training  system, 
including  classroom  lecture,  computer  based  training,  and  two  highly  advanced,  state-of- 
the-art  simulators.  The  most  advanced  of  these  flight  simulators  introduces  visual 
simulation  into  the  training  pipeline. 

The  T-45  Operational  Flight  Training  (OFT)  simulator  is  built  around  a  replica  of 
a  Goshawk  cockpit.  A  Gould  SEL  32/8780  computer  with  2M  bytes  of  memory  controls 
the  simulation  scenario.  While  anchored  to  a  fixed  platform,  the  simulator  uses  G-seat/G- 
motion  cueing  systems  and  an  aural  system  to  provide  realistic  aircraft  motion,  audio  and 
aerodynamic  cues.  The  visual  simulation  is  projected  onto  a  partial  dome  screen, 
allowing  the  student  a  1 80  degree  horizontal  and  60  degree  vertical  field  of  view.  The 
three  areas  in  the  training  curriculum  that  require  the  greatest  use  of  the  SNA’s  visual 
skills  are  landing  skills,  weapons  training  and  carrier  qualifications.  The  visual  system 
allows  extensive  training  in  all  of  these  areas.  For  weapons  training,  the  simulator 
includes  air/ground  bomb,  racket  and  strafing  models,  rocket  models,  and  the  dust  puffs 
of  ordnance  exploding  on  the  ground  at  the  impact  points.  For  landing  skills,  several 
highly  detailed  models  of  Naval  Air  Stations,  outlying  fields  and  Air  Force  Bases  are  also 
modeled.  Finally,  for  carrier  operations,  a  dynamic  model  of  the  aircraft  carrier  USS 
NIMITZ  is  provided.  In  the  simulator,  the  SNA  can  experience  carrier  deck  arrested 
landings,  on  deck  taxiing,  and  catapult  takeoffs  in  sea  states  from  zero  to  sea  state  five. 
Rain,  fog,  cloud  cover,  sea  state,  and  turbulence  can  be  introduced,  modified,  or  dismissed 
by  simple  instructor-computer  interface  controls  in  order  to  provide  a  more  realistic 
aircraft  flight  deck  environment  (T-45A  Simulator  Training  System  Overview,  1-3  - 1-4). 

C.  VISUAL  SIMULATOR  EFFECTIVENESS 

The  training  effectiveness  of  visual  dome  simulators  is  hard  to  measure.  One  means 
of  measuring  simulator  effectiveness  is  through  transfer  of  training:  the  ability  of  the 
student  to  transfer  what  he  has  practiced  in  the  simulator  to  the  actual  aircraft  (Lintem 
et  al.  1990,  320).  But  the  skills  trained  in  a  simulator  are  controlled  and  constrained  by 
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the  simulator's  characteristics.  These  characteristics  include  realism  of  the  cockpit  and 
instrument  panel,  realism  of  motion  effects,  and  in  the  case  of  visual  simulators,  realism 
of  the  visual  system. 

Even  if  transfer  of  training  is  verified,  singling  out  the  individual  role  played  by  a 
simulator  characteristic  can  be  difficult.  At  the  University  of  Illinois,  researchers  found 
that  beginning  flight  students  were  positively  affected  by  visual  simulator  landing  practice 
prior  to  solo  landings  in  actual  aircraft  (Lintem  et  al.  1990,  324).  While  this  study 
indicates  that  visual  simulators  have  a  positive  training  effect,  how  much  of  the  training 
transfer  can  be  attributed  to  the  simulator's  visual  system  is  hard  to  measure.  Student 
motivation  and  the  additional  training  with  similar  controls  may  also  contribute  to  the 
improved  flight  performance  (Gopher  et  al.  1994, 401).  In  an  attempt  to  isolate  the  effect 
of  a  visual  system,  various  studies  have  examined  how  much  visual  acuity  is  required 
to  achieve  a  transfer  of  training.  Detail  and  field  of  view  may  also  play  an  important 
part  in  the  transfer  of  training.  Density,  or  the  visual  definition  of  a  simulated  object, 
appears  to  be  more  important  than  the  visual  detail  of  the  object  (Kleiss  and  Hubbard 
1993,  653).  Concurrent  with  the  development  of  the  T-45  simulator  and  training 
program,  D.  P.  Westra  and  colleagues  conducted  a  study  for  the  Naval  Training  Systems 
Center  in  Orlando,  Florida  (Westra  et  al.  1985).  The  Westra  study  provided  key 
recommendations  for  the  design  of  carrier  landing  tasks  for  the  T-45  visual  simulator. 
In  the  study  one  group  of  SNA's  received  training  in  an  experimental  visual  simulator  and 
were  compared  against  a  control  group  of  SNA's  who  did  not  receive  training  in  the 
simulator.  Special  equipment  was  developed  to  measure  aircraft  deviations  as  the  student 
approached  the  landing  field.  Westra's  study  is  important  because  it  identified  a 
measurable  improvement  in  Flight  Carrier  Landing  Practice  (FCLP)  performance  when 
student  pilots  practiced  with  the  simulator.  However,  Westra  did  not  extend  his  research 
to  the  deck  of  an  actual  carrier  because  of  difficulties  with  stabilizing  his  equipment  on 
a  rolling  deck. 
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D.  PROBLEM  STATEMENT 

The  U.S.  Navy  JUTP  is  conducted  in  two  locations,  Meridian,  Mississippi  and 
Kingsville,  Texas.  The  T-45  visual-dome  simulator  was  installed  in  Kingsville  in  1990. 
From  1990  to  1993  the  T-45  A  Goshawk  was  not  ready  for  students.  Instructor  training 
for  the  T-45  A  began  in  the  summer  of  1993  and  students  started  actual  T-45  A  flights 
nearly  a  year  later.  However,  while  the  simulator  waited  in  Kingsville,  T-2C  Buckeye 
students  were  exposed  to  demonstrations  of  the  flight  simulator  prior  to  their  carrier 
qualifications  and  weapons  training.  The  T-2C  students  in  Meridian  did  not  receive  visual 
simulator  exposure.  By  comparing  these  two  groups  in  Meridian  and  Kingsville,  this 
study  attempts  to  quantify  and  analyze  the  effectiveness  of  the  new  visual  simulator. 
Chapter  II  describes  in  detail  carrier  qualification  requirements,  the  data,  and  the  measures 
of  effectiveness  used  for  the  study.  The  results  of  the 
Chapter  III.  Discussion  of  the  results  and  conclusions 
Chapters  IV  and  V,  respectively. 


data  analysis  are  presented  in 
of  the  study  are  addressed  in 
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II.  METHODOLOGY 


A.  CARRIER  QUALIFICATION  REQUIREMENTS 

Student  Naval  Aviators  reviewed  for  this  thesis  were  in  the  intermediate  phase  of 
their  strike  flight  training.  Previous  training  included  basic  ground  school  and  basic  flight 
skills  in  the  T-34  turbo  prop  training  aircraft.  Students  receive  carrier  qualification 
instruction  as  the  last  section  of  their  intermediate  syllabus.  Previous  sections  in  the 
syllabus  include  Familiarization,  Basic  Instruments,  Radio  Instruments,  Airways 

Navigation,  Basic  Formation,  Night  Fly,  Gunnery/Weapons,  and  Out  of  Control  Flight. 

1.  Field  Carrier  Landing  Practice  (FCLP) 

During  carrier  qualification  training,  the  student  receives  1 1  periods  of  instruction 
which  are  coded  CQ-1  through  CQ-llx.  CQ-1  begins  with  a  basic  review  of  procedures 
and  flight  skills  to  allow  both  the  student  and  the  instructor  to  adapt  to  each  other.  The 
student  is  assigned  to  the  same  instructor  throughout  the  CQ  stage.  The  instructor  is  a 
qualified  Landing  Signal  Officer  (LSO)  trained  and  certified  to  assist  pilots  as  they 
approach  and  land  on  the  deck  of  a  carrier.  During  CQ-1  through  CQ-10,  the  student 
flies  Field  Carrier  Landing  Practice  (FCLP)  landings  on  an  air  strip,  which  is  painted  to 
resemble  the  deck  of  an  aircraft  carrier.  A  Fresnel  Lens  Optical  Landing  System 
(FLOLS),  identical  to  the  FLOLS  on  a  carrier,  is  used  to  help  the  student  line  up 
properly.*  The  instructor/LSO  guides  the  student  to  the  airstrip  using  the  same  directions 
that  will  be  used  on  the  aircraft  carrier.  At  the  end  of  CQ-1,  CQ-2,  CQ-9,  and  CQ-10, 
the  student  is  evaluated  by  his  instructor  using  an  Aviation  Training  Form  (ATF).  A  grade 
is  assigned  based,  in  part,  on  an  evaluation  of  the  student's  headwork,  airwork,  start 
position,  line  up,  and  speed  control.  A  full  listing  of  the  performance  areas  considered 
on  these  ATF's  is  presented  in  Appendix  A.  Grades  are  given  on  a  four  point  interval 
scale  with  1  =  unsatisfactory,  2  =  below  average,  3  =  average,  4  =  above  average.  The 


*  The  FLOLS  is  sometimes  referred  to  as  the  glideslope  indicator  or  in  pilot 
vernacular  "the  ball". 
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points  are  summed  and  averaged  for  a  final  ATF  score.  At  the  end  of  CQ-10,  the  student 
is  evaluated  for  field  qualification.  If  field  qualification  is  granted,  the  student  advances 

to  CQ-11,  which  includes  the  actual  carrier  landing. 

2.  Carrier  Qualification 

For  his  carrier  qualification,  the  Student  Naval  Aviator  receives  two  numeric  grades, 
an  overall  flight  performance  grade  and  a  LSO  trend  analysis  grade  of  his  actual 
approaches  and  arrests  on  the  carrier  deck.  The  LS 0/instructor  evaluates  the  Student 
Naval  Aviator's  performance  in  nineteen  areas  listed  on  the  Aviation  Training  Form 
(ATF)  for  CQ-1 1  as  either  Unsatisfactory,  Below  Average,  Average,  and  Above  Average. 
While  the  LSO's  evaluation  of  these  performance  areas  is  subjective,  his  own  flight 
experience  as  well  as  standardized  instructor  training,  lend  validity  to  the  assigned  grade. 

In  addition  to  the  specific  items  listed  on  the  CQ  ATF,  the  LSO  must  also  consider 
the  following  criteria  for  qualification: 

■  Student  displays  no  dangerous  tendencies. 

■  Student  demonstrates  steady  or  improved  performance  during  FCLP/ship 
qualification  period. 

■  Student  requires  minimum  LSO  assistance  during  the  final  two 
approaches/landings. 

■  Student  is  predictable-ready  for  Advanced  Strike. 

■  Student  has  a  50  percent  or  better  boarding  rate  (Chief  of  Naval  Air  Training 
(CNATRA)  CQ  instruction  1994,  III-3). 

The  second  numerical  grade  is  reported  on  the  LSO  Trend  Analysis  Sheet.  Every 
pass  and  approach  that  a  Student  Naval  Aviator  makes  on  the  carrier  is  graded  by  his 
LSO.  This  practice  is  continued  even  after  the  SNA  completes  his  training  and  enters  a 
Fleet  Squadron.  The  student  is  required  to  make  at  least  six  approaches  to  the  aircraft 
carrier;  four  of  the  approaches  must  be  complete,  arrested  landings  and  two  of  the 
approaches  must  be  touch  and  go  landings.  Each  approach  to  the  carrier  is  assigned  a 
grade  on  a  five  point  scale.  Once  again  the  grades  are  subjective  but  are  based  on  the 
LSO's  experience  observing  and  waving  other  pilot  approaches.  Westra  writes  that 
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previous  research  has  identified  four  factors  that  the  LSO  considers  when  assigning  the 
trend  analysis  scores.  These  four  factors,  in  order  of  importance  are:  1)  touchdown 
accuracy,  2)  approach  glideslope  control,  3)  approach  lineup  control,  and  4)  approach 
angle  of  attack  control  (Westra  et  al.  1985,  41).  The  LSO  marks  and  grades  are  listed  in 
Appendix  B.  The  minimum  grade  acceptable  for  the  LSO  Trend  Analysis  is  2.4. 
Qualification  of  students  with  less  than  2.4  may  be  given  in  special  cases,  but  only  if  the 
student  demonstrates  an  overall  improving  trend.  Additionally,  a  student  with  2.4  or 
better  may  be  disqualified  for  not  meeting  any  of  the  criteria  scored  on  the  ATF. 

B.  DATA  DESCRIPTION 

Data  were  collected  for  SNA's  at  the  CNATRA  archives  at  Naval  Air  Station  (NAS) 
Corpus  Christi,  Texas  for  the  carrier  qualification  periods  in  May  and  July  of  1993. 

1.  Boat  Detachment  Dates 

Carrier  qualification  flights  were  conducted  between  May  14-19  and  July  23-27, 
1993.  USS  AMERICA  (CV  66)  and  USS  JOHN  F.  KENNEDY  (CV  67)  were  the 
designated  carriers  for  May  and  July,  respectively.  Both  carriers  were  located  off  the 
eastern  coast  of  Florida.  Students  and  their  instructors  flew  their  aircraft  from  Key  West, 
Florida  and  met  the  carriers  at  sea.  The  weather  during  these  qualification  periods 
appeared  mostly  mild,  with  sea  state  ranging  from  1/2  to  2  meters,  winds  5-22  knots,  and 
a  cloud  ceiling  averaging  around  3500  feet  (Fleet  Numeric  Meteorological  and 
Oceanographic  Center,  1993).  A  summary  listing  of  the  weather  conditions  is  provided 
in  Appendix  C. 

2.  Training  Squadrons 

a.  VT-I9,  NAS  Meridian,  Mississippi 

The  students  at  VT-19  did  not  receive  exposure  to  the  T-45  simulator. 
Student  scores  from  the  May  and  July  carrier  qualification  periods  of  1993  were  analyzed. 
Of  the  79  students  who  participated  in  the  two  Carrier  Qualification  periods,  69  student 
records  were  available  for  examination. 
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b.  VT-23,  NAS  Kingsville,  Texas 

Documentation  indicates  that  75  T-2  students  in  Kingsville  received  exposure 
to  the  T-45  simulator  prior  to  their  carrier  qualifications  during  two  separate  carrier 
qualification  periods  in  May  and  July  of  1993.  Exposure  to  the  simulator  consisted  of  a 
90  minute  demonstration  of  carrier  approaches,  arrests,  and  touch  and  goes.  Each  student 
sat  in  the  cockpit  of  the  simulator  for  approximately  5-10  minutes  and  observed  the 
aircraft  fly  from  approximately  three  miles  aft  of  the  carrier,  through  the  approach  pattern, 
to  the  deck  for  a  touch  and  go,  and  back  around  the  pattern  again  for  an  actual  arrested 
landing.  Students  did  not  actually  fly  the  simulator  because  the  controls  did  not  match 
those  of  the  A-4,  but  the  students  could  hold  the  stick  and  feel  the  aircraft  movements  and 
power  adjustments.  The  instructor  monitoring  the  demonstration  could  stop  the  simulation 
at  any  time  for  questions  or  observations.  Sixty-seven  student  records  were  available. 
A  breakdown  of  the  number  of  students  from  each  carrier  qualification  period  and 
squadron  is  listed  in  Table  1. 


Table  1:  Number  of  students  participating  during  each  carrier  qualification  versus 
number  of  students  reviewed. 


Squadron 

Month 

Total  Students  Present 

Number 

Reviewed 

Percentage 

Reviewed 

VT-19 

May 

40 

37 

92.5 

VT-23 

May 

43 

41 

95.3 

VT-19 

July 

39 

32 

82.1 

VT-23 

July 

32 

26 

81.3 

3.  The  Landing  Signal  Officer  (LSO) 

For  their  carrier  qualification  instruction,  students  are  assigned  to  a  Landing  Signal 
Officer  (LSO),  a  pilot  certified  to  provide  directions  to  other  pilots  during  their  carrier 
approach  and  landing.  LSO's  are  trained  in  accordance  with  the  LSO  NATOPS.  While 
assigned  to  his/her  first  squadron,  a  pilot  may  begin  the  LSO  qualification  process.  First 
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the  pilot  attends  a  three  week  LSO  school  in  Oceana,  Virginia.  Upon  completion  of  the 
school,  the  pilot  returns  to  his  squadron  and  begins  an  on-the-job-training  qualification 
process,  beginning  with  the  Field  Qualification.  The  Field  Qualification  allows  the  pilot 
to  direct  other  pilots  flying  his  own  type  of  aircraft  to  land  on  an  airfield  when  a  fully 
qualified  LSO  is  present.  The  second  qualification,  the  Squadron  Qualification,  allows 
the  pilot  to  land  any  type  of  aircraft  on  the  deck  of  an  aircraft  carrier  during  periods  of 
good  weather.  The  final  qualification,  the  Wing  Qualification,  permits  the  pilot  to  land 
any  aircraft  on  the  deck  of  an  aircraft  carrier,  during  day  or  night.  When  the  pilot 
transfers  from  the  squadron  to  a  training  squadron,  he/she  must  be  either  Squadron  or 
Wing  Qualified  as  an  LSO.  The  pilot  attends  a  three  day  refresher  LSO  school  and  must 
attend  at  least  three  carrier  qualification  periods  before  being  designated  Training  LSO. 

In  the  training  squadron,  the  LSO  instructs,  evaluates,  and  grades  the  student 
through  out  the  entire  carrier  qualification  period.  Should  a  student  disqualify  during  the 
carrier  qualification  period,  he  is  reassigned  to  a  new  LSO  and  must  repeat  all  previous 
carrier  qualification  lessons.  The  students  examined  in  this  study  were  graded  by  12 
different  LSO's,  5  LSO's  from  VT-19  and  7  LSO's  from  VT-23.  A  breakdown  of  the 
LSO's  and  the  number  of  their  students  reviewed  are  listed  in  Table  2.  To  protect 
confidentiality,  LSO's  are  referred  to  alphabetically. 

C.  MEASURES  OF  EFFECTIVENESS 

In  comparing  the  two  squadrons,  VT-19  and  VT-23,  several  measures  of 
effectiveness  (MOE)  were  used  to  determine  if  the  T-45  visual  simulator  had  an  effect 
on  SNA  carrier  qualification.  The  first  MOE  looked  at  the  percentage  of  disqualifications 
within  each  squadron.  The  second  MOE  examined  the  final  LSO  Trend  Analysis  grade 
for  each  SNA.  The  third  MOE  examined  the  final  ATF  performance  evaluation  for  each 
SNA.  Finally,  the  CQ-llx  ATF  was  broken  down  into  its  separate  19  performance  areas 
to  determine  if  there  was  a  difference  in  certain  areas  of  flight  performance. 
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Table  2:  LSO  breakdown  by  squadron  and  carrier  qualification  period. 


LSO 

Squadron 
VT-19  VT-23 

Month 

May 

July 

Number  of  Students 
May  July 

A 

X 

X 

X 

7 

9 

B 

X 

X 

X 

8 

10 

C 

X 

X 

X 

7 

4 

D 

X 

X 

7 

E 

X 

X 

X 

8 

9 

F 

X 

X 

X 

9 

4 

G 

X 

X 

X 

6 

5 

H 

X 

X 

X 

9 

4 

I 

X 

X 

X 

9 

3 

J 

X 

X 

8 

K 

X 

X 

5 

L 

X 

X 

5 

12 


III.  RESULTS 


A.  COMPARISON  OF  DISQUALIFICATION  RATES 

Table  3  presents  the  number  of  disqualifications  (DQ)  per  squadron  during  each 
carrier  qualification  period. 


Table  3:  Disqualification  rates  by  squadron  and  carrier  qualification  period. 


Squadron 

Number 

Qualified 

May 

Number 

Disqualified 

Percent 

Disqualified 

Number 

Qualified 

July 

Number 

Disqualified 

Percent 

Disqualified 

VT-19 

40 

4 

10.0 

36 

2 

5.2 

VT-23 

49 

3 

7.0 

29 

3 

9.3 

To  compare  the  two  squadrons,  a  two  by  two  contingency  table  was  developed  by 
counting  the  number  of  disqualifications  and  qualifications  within  each  squadron  during 
a  particular  qualification  period.  A  contingency  table  is  used  to  count  the  number  of 
occurrences  and  compare  that  count  to  the  expected  number  of  occurrences  based  on  the 
assumption  that  the  DQ  rate  is  the  same  for  both  groups.  The  difference  between  the 
actual  and  expected  occurrences  is  used  to  generate  a  chi-square  number.  Large  chi- 
square  values  indicate  large  differences  between  the  actual  and  expected  occurrences.  For 
this  comparison,  four  two  by  two  contingency  tables  were  used.  Two  tables  were  used 
to  test  the  hypothesis  that  the  number  of  disqualifications  was  independent  of  the 
squadron  during  the  qualifications  periods  of  May  (Table  I)  and  July  (Table  II).  The  two 
remaining  tables  were  used  to  test  the  hypothesis  that  the  number  of  disqualifications  was 
independent  of  the  qualification  period  for  VT-19  (Table  III)  and  VT-23  (Table  IV).  The 
final  contingency  table  (Table  V)  pools  squadrons  and  qualification  period  together  into 
a  two  by  four  table,  again  testing  for  independence  in  the  number  of  disqualifications. 
Table  4  lists  the  results  of  the  five  contingency  tables.  Judging  from  the  very  low  chi- 
square  values  and  corresponding  high  p-values,  there  is  no  evidence  to  say  that  the 
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number  of  disqualifications  is  independent  of  squadron  and  carrier  qualification  month. 
The  squadrons  appear  to  disqualify  equal  number  of  students.  Complete  contingency 
tables  are  presented  in  Appendix  D. 


Table  4:  Contingency  Table  results  for  comparison  of  disqualification  (DQ) 
counts. 


Contingency  Table 

Tested  for  Independence 

DF 

Chi-Square 

P< 

I 

May  DQ's 

1 

0.131 

0,7174 

II 

JulyDQ's 

1 

0.443 

0.5057 

III 

VT- 1 9  DQ's 

1 

0.441 

0.5066 

IV 

VT-23  DQ's 

1 

0.143 

0.7053 

V 

All  DQ's 

3 

0.786 

0.8528 

B.  COMPARISON  OF  LSO  TREND  ANALYSIS  SCORES 

The  LSO  evaluates  every  approach  the  student  flies  on  the  carrier  using  a  seven 
point  scale  labelled  from  zero  to  five,  where  zero  indicates  a  cut  or  an  unsafe  pass  and 
five  indicates  a  perfect  pass.  Table  5  lists  the  grading  scale  used  by  the  LSO  for  the 
carrier  approach  evaluation.  Students  are  required  to  make  at  least  six  passes  (four  arrests 
and  two  touch  and  gos).  At  the  end  of  the  qualification  period  the  student's  approach 
grades  are  averaged  to  two  decimal  places  to  form  the  student's  LSO  Trend  Analysis 
Score.  Four  of  the  students  reviewed  for  this  study  did  not  have  a  complete  LSO  Trend 
Analysis  form  in  their  training  record.  Their  LSO  Trend  Analysis  records  are  not 
reviewed  in  the  following  comparison. 


Table  5:  LSO  Trend  Analysis  grading  scale 


Symbol 

Definition 

Grade 

OK 

Perfect  pass. 

5 

OK 

Reasonable  devations  with  good  corrections. 

4 

(OK) 

Fair,  reasonable  deviations. 

3 

B 

Bolter. 

2.5 

- 

Below  average  but  safe  pass. 

2 

PWO 

Power  waveofif 

2 

owo 

Own  waveofif. 

2 

wo 

Waveofif 

1 

c 

Cut.  Unsafe,  Gross  deviations  made  inside  waveoff  window. 

0 
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The  LSO's  from  a  given  squadron  graded  only  students  from  that  squadron. 
Additionally,  each  student  was  graded  by  the  same  LSO  during  the  entire  carrier 
qualification  process.  Because  12  different  LSO's  graded  students  during  this  time  period, 
consideration  must  be  given  to  individual  grading  differences  between  the  LSO's.  Table 
6  presents  the  breakdown  of  the  LSO's  used  during  a  qualification  period  by  squadron. 
Note  that  not  all  the  LSO's  grading  in  May,  graded  in  July.  Likewise  additional  LSO's 
were  present  in  July  who  did  not  grade  in  May.  In  constructing  a  model  for  comparison 
of  the  two  squadrons,  it  is  necessary  to  make  assumptions  about  the  LSO's  employed  in 
grading  SNA's.  The  analyses  presented  are  essentially  based  on  the  concept  of  a  pool  of 
trained  LSO's,  any  of  whom  might  be  assigned  to  a  particular  squadron  (and  set  of 
students). 


Table  6:  Breakdown  of  LSO's  used  by  each  squadron  during  a  particular 
qualification  period. 


S  quadro n 

VT-19 

VT-23 

Month  Total 

M  ay  LSO's 

A,  B  ,  C  ,  D  ,  E 

F,G,H,I,J 

10 

July  LSO's 

A  ,B  ,C  ,E 

F,G,H,I,K,L 

10 

Squadron  Total 

5 

7 

Figure  1  presents  box  plots  for  the  LSO  trend  analysis  scores  given  by  individual 
LSO's  during  May  and  July  qualification  periods.  The  box  plots  in  this  graph  present 
important  information  on  the  distribution  of  the  LSO's  trend  analysis  scores.  The  top  and 
bottom  line  of  the  box  represent  the  first  and  third  quartiles  of  the  range  for  the  LSO's 
trend  analysis  grades.  Fifty  percent  of  the  data  is  located  within  the  box.  The  line  within 
the  box  is  the  median  trend  analysis  score  given  by  that  LSO.  In  some  cases,  the  LSO 
gave  the  same  grade  to  several  students  and  the  median  trend  analysis  score  may  be 
located  on  the  top  or  bottom  of  the  box.  Median  values  are  labeled  on  the  boxes  to  aid 
in  their  recognition.  Lines  extending  from  either  end  of  the  box  indicate  the  shape  of  the 
distribution's  tail.  Long  lines  indicate  a  stretched  distribution,  short  lines  indicate 
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compactness  and  bunched  data.  Asterisks  represent  outliers,  data  points  that  differ 
significantly  from  the  rest  of  the  data  (Chambers  et  al.  1983,  22).  The  width  of  each  box 
plot  is  proportional  to  the  square  root  of  the  sample  size  of  the  data  used  to  plot  the  box. 
To  aid  in  the  interpretation  of  sample  size,  the  actual  sample  size  appears  as  a  number 
beneath  each  box  plot. 

As  can  be  seen  from  Figure  1,  there  appears  to  be  very  little  difference  in  the 
median  trend  analysis  score  given  by  the  LSO's  during  the  May  qualification  period. 
Medians  range  from  approximately  2.43  for  LSO  C  to  2.81  for  LSO  J.  The  relatively 
short  median  range  of  0.38  indicates  little  variance  between  the  LSO's.  However,  VT-19 
has  three  low  outliers  and  VT-23  has  a  one  low  and  one  high  outlier.  In  all  cases,  a  low 
outlier  corresponds  to  a  student  who  ultimately  was  disqualified  during  that  qualification 
period.  The  minimum  and  maximum  median  for  July  are  2.5  for  LSO’s  E,  F,  and  I  and 
2.65  for  LSO  L.  Both  bounds  occur  for  LSO's  in  VT-23.  However,  the  range  of  medians 
in  July  is  considerably  smaller  than  the  range  of  medians  in  May,  0.15  vice  0.31.  The 
lack  of  outliers  in  July  is  probably  due  to  the  data  reviewed  rather  than  an  actual  grading 
difference.  Students  who  disqualified  in  July  were  given  a  second  opportunity  to  qualify 
in  September.  Records  were  not  reviewed  for  September,  hence  the  July  LSO  Trend 
Analysis  Sheet  recording  the  disqualification  were  not  reviewed  as  well. 

When  the  trend  analysis  scores  are  combined  across  both  qualification  periods. 
Figure  2  results.  Figure  2  illustrates  the  trend  analysis  scores  for  the  LSO's.  The  median 
range  extends  from  2.44  to  2.81.  VT-23  has  two  additional  outliers,  one  high  and  one 
low,  for  a  total  of  four  outliers.  VT-19  still  has  the  three  low  outliers.  For  the  most  part 
however,  there  appears  to  be  little  variance  in  the  LSO's  grading  procedures.  This 
observation  can  be  seen  more  clearly  in  Figure  3,  which  presents  box  plots  of  the 
squadron's  trend  analysis  scores  over  both  qualification  periods.  Even  with  the  presence 
of  the  low  outliers  in  VT-19,  the  box  plots  are  remarkably  similar.  The  overall  median 
trend  analysis  score  for  the  two  squadrons  is  only  .01.  As  indicated  by  the  above  figures, 
there  appears  to  be  little  difference  in  the  LSO  trend  analysis  scores  in  VT-19  and  V-23. 
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Figure  1:  Box  Plots  of  LSO  Trend  Aniaysis  Scores 


LSO  Trend  Analysis,  Both  Qualfication  Periods 


Figure  2:  Box  Plots  of  LSO  Trend  Anlaysis  Scores,  Both  Qualification 
Periods. 


Squadron  LSO  Trend  Analysis 


Figure  3:  Box  Plots  of  Squadron  LSO  Trend  Analysis  Scores,  Both 
qualification  periods. 
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To  quantify  the  box  plot  comparisons,  an  Analysis  of  Variance  (ANOVA)  was 
conducted  to  test  for  significant  differences  between  the  squadrons.  An  ANOVA  can  be 
used  to  test  for  the  equality  of  two  or  more  means  that  come  from  independent,  normal 
populations  with  equal  variances.  It  should  be  noted  however,  that  the  primary  use  of  an 
ANOVA  is  to  model  data  from  a  designed  experiment  in  which  certain  factors  are 
controlled  while  others  vary.  As  the  data  for  this  study  is  historical,  the  ANOVA  model 
may  oversimplify  any  relationships  in  the  data. 

As  mentioned  earlier,  the  particular  LSO's  assigned  to  a  given  squadron  at  a  given 
time  have  received  the  same  training  and  are  selected  haphazardly,  with  no  special 
purpose.  (The  same  is  true  of  the  student  aviators.)  The  statistical  model  employed  for 
the  following  ANOVA  discussions  is  aimed  at  identifying  possible  differences  between 
the  two  squadrons  undergoing  carrier  qualification;  any  identified  difference  may  then  be 
attributed  to  the  use  of  the  simulator.  The  individual  trend  analysis  score  received  by  any 
student  is  assumed  to  be  an  accumulation  of  effects  of  (a)  the  squadron  trained  in 
(simulator  or  no  simulator),  (b)  the  month  of  qualification  (May  or  July),  (c)  possible 
interaction  between  squadron  and  month,  and  (d)  the  particular  LSO  grading  the  given 
student.  In  statistical  jargon,  the  squadron  and  month  contributions  are  fixed  effects, 
while  the  LSO's  are  random  effects  and  nested  within  squadrons.  An  implication  of  these 
assumptions  then  is  that  (possible)  differences  in  the  fixed  effects  are  judged  relative  to 
the  variability  between  the  LSO's  within  the  squadron  (as  opposed  to  the  variability  within 
scores  issued  by  the  same  LSO). 

The  currently  available  statistical  packages  do  not  allow  for  nested  factors  with 
varying  numbers  of  levels  (in  May  5  LSO's  were  used  in  each  squadron,  while  in  July 
there  were  4  LSO's  used  by  VT-19  and  6  LSO's  used  by  VT-23).  The  ANOVA  results 
in  Table  7  were  computed  in  APL  and  (partially)  verified  by  the  MTNITAB®  program. 
The  complete  model  is  presented  in  Appendix  E. 
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The  results  confirm  the  prediction  of  the  box  plots.  Squadron  and  qualification 
period  have  no  significant  effect  on  the  LSO  trend  analysis  scores.  The  hypothesis  that 
the  mean  trend  analysis  scores  are  equal  between  the  squadrons  and  qualification  period 
cannot  be  rejected.  Likewise,  there  appears  to  be  no  significant  interaction  between  the 
squadrons  and  qualification  period.  While  there  does  appear  to  be  some  variance  between 
the  LSO's  within  a  squadron,  the  variance  is  not  statistically  significant  with  a  =  0.05. 
If  the  exposure  to  the  visual  simulator  had  any  effect  on  the  students  in  VT-23,  its  effects 
are  not  apparent  in  the  the  trend  analysis  scores. 


Table  7:  ANOVA  Results  for  LSO  Trend  Analysis  Scores. 


Effect 

DF 

Sum  Squares 

Mean  Squares 

F-value 

p-value 

Squadron 

1 

0.1943* 

0.1943 

2.41 

0.151 

Qualification  Period 

1 

0.0310* 

0.0310 

0.39 

0.549 

Interaction 

1 

0.0246* 

0.0246 

0.31 

0.592 

LSO  Variance 

10 

0.8049* 

0.08049 

1.72 

0.066 

Error 

118 

5.2556 

0.0454 

Total 

131 

6.4548 

*Adjusted 

C.  COMPARISON  OF  LSO  PERFORMANCE  EVALUATION 

In  addition  to  the  Trend  Analysis,  the  LSO  also  grades  the  student  on  19  different 
performance  areas  listed  on  ATF  CQ-1  lx.  These  performance  areas  consist  of  headwork, 
procedure,  airwork,  pattern,  start  position,  speed  control,  attitude  control,  power  control, 
line  up,  glideslope  control,  error  detection/correction,  waveoff  technique,  touch  and 
go/bolter  technique,  response  to  LSO,  formation/pattem  entry,  radio  procedures,  fuel 
management,  deck  procedures,  and  catapult  procedures.  Each  of  these  19  areas  receives 
a  mark  of  unsatisfactory,  below  average,  average,  or  above  average.  The  marks  are  then 
converted  to  an  ordinal  scale  with  "1"  at  unsatisfactory  and  "4"  at  above  average.  The 
LSO  performance  evaluation  is  the  average  of  the  grades  assigned  to  the  19  performance 
areas.  LSO  performance  evaluations  will  be  compared  in  a  similar  manner  to  the  LSO 
trend  analysis  scores. 


Figure  4  presents  box  plots  for  the  LSO  performance  evaluations  given  by 
individual  LSO's  during  May  and  July  qualification  periods.^  During  the  May 
qualification  period,  it  is  readily  apparent  that  the  LSO's  in  VT-19  gave  lower 
performance  evaluations  than  the  LSO's  in  VT-23.  The  range  of  the  median  is  2.89  to 
3.05,  with  the  low  median  in  VT-19  and  the  high  median  in  VT-23.  Within  the 
squadrons,  both  VT-19  and  VT-23  have  a  median  range  of  0.105.  Such  a  small  range 
seems  to  indicate  little  difference  between  the  individual  LSO's.  Of  the  ten  LSO's,  five 
have  a  median  of  3.0,  yet  another  indication  of  similarity.  However,  the  distribution  of 
the  performance  evaluations  across  all  LSO's  and  across  the  individual  squadrons  appear 
quite  different.  In  spite  of  the  five  3.0  medians,  the  majority  of  performance  evaluations 
in  VT-19  occur  below  3.0  and  the  majority  of  performance  evaluations  in  VT-23  occur 
above  3.0.  The  July  qualification  presents  a  similar  picture.  The  range  of  the  median 
extends  from  2.95  to  3.05.  Within  the  squadrons,  there  appears  more  similarity  between 
the  LSO's,  both  in  their  median  performance  evaluations  and  the  distribution  of  the 
grades.  Note  that  of  the  ten  LSO's  who  graded  during  this  period,  six  had  median 
performance  evaluations  at  3.0.  However,  VT-19  still  appears  to  have  lower  performance 
evaluations  than  VT-23.  The  majority  of  VT-19  performance  evaluations  occur  below  3.0 
and  the  two  lowest  medians  occur  in  this  squadron.  VT-23  performance  evaluation 
appear  more  equally  distributed  about  3.0.  In  fact  of  the  six  LSO's  in  VT-23,  four  have 
medians  at  3.0,  and  the  other  two  LSO's  are  split  about  the  3.0  mark. 

The  difference  between  the  squadrons  is  more  pronounced  when  the  performance 
evaluations  are  examined  over  both  qualification  periods,  as  illustrated  in  Figure  5.  The 
range  of  the  median  extends  from  2.89  to  3.05.  While  a  range  of  0.16  seems  relatively 
insignificant,  the  distribution  of  the  grades,  belies  the  small  range.  Seventy-five  percent 
of  the  grades,  for  all  LSO's  in  VT-19,  are  at  or  below  3.0.  In  VT-23,  the  grades  are  more 
appear  more  equally  distributed  about  3.0.  Figure  6  presents  box  plots  of  the  squadron 


^Recall  that  two  additional  students  were  reviewed  for  this  comparison.  Hence,  the 
sample  sizes  will  not  match  the  sample  sizes  for  the  May  LSO  Trend  Analysis 
Comparison. 
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Q-1  lx  Performance  Evaluation 


zz 


Figure  4:  Box  Plots  of  CQ-llx  Average  Performance  Evaluations. 


CQ-1 IX  Performance  Evaluation,  Both  Qualification  Periods 


Figure  5:  Box  Plots  of  Individual  LSO  Performance  Evaluations,  Both 
qualification  periods. 


Squadron  CQ-1  IX  Performance  Evaluation 
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Figure  6:  Box  Plots  of  Squadron  Performance  Evaluations,  Both 
qualification  periods. 
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performance  evaluations.  The  difference  in  this  graph  is  even  more  pronounced.  With  the 
exception  of  the  three  outliers,  all  of  the  grades  in  VT-23  are  at  the  median  of  3.0.^  The 
grades  in  VT-19  have  greater  variation  and  are  centered  on  the  median  of  2.95.  Figures 
4-6  seem  to  indicate  that  there  is  difference  in  the  CQ-llx  performance  evaluations 
between  VT-19  and  VT-23.  Notably,  students  in  VT-23  appear  to  have  higher 
performance  evaluations. 

An  ANOVA  similar  to  the  model  used  for  the  LSO  trend  analysis  was  developed 
to  further  examine  the  difference  between  the  squadron  performance  evaluations.  The 
hypothesis  to  be  tested  assumes  that  the  mean  performance  evaluation  is  equal  between 
the  squadrons  and  qualification  period  and  that  there  is  no  interaction  between  squadron 
and  qualification  period  or  extreme  variance  between  the  LSO's.  Results  of  this  analysis 
are  presented  in  Table  8.  As  predicted  by  the  box  plots,  there  is  a  significant  difference 
between  the  squadrons.  However,  there  is  no  significant  difference  between  the 
qualification  periods  nor  is  there  any  interaction  between  the  squadrons  qualification 
periods.  Between  the  LSO's  there  is  a  significant  difference  in  the  variation  of  their 
grading  procedures.  Because  the  squadron's  used  their  own  LSO's,  it  impossible  to 


Table  8:  ANOVA  Results  for  CQ-llx  Average  Performance  Evaluations. 


Effect 

DF 

Sura  Squares 

Mean  Squares 

F- value 

p-value 

Squadron 

1 

0.0736* 

0.0736 

12.23 

0.006 

Qualification  Period 

1 

0.0056* 

0.0056 

0.93 

0.358 

Interaction 

1 

0.0016* 

0.0016 

0.26 

0.619 

LSO  Variance 

10 

0.0601* 

0.0601 

3.12 

0.001 

Error 

122 

0.2353 

0.0019 

Total 

136 

0.3653 

*Adjusted 

^ While  it  appears  that  all  but  three  performance  evaluations  is  3.0,  the  previous  box 
plots  have  shown  that  this  is  not  the  case.  The  outliers  in  this  graph  correspond  to  more 
than  one,  and  in  most  instances,  several  grades  at  2.95,  3.05,  and  2.10.  However,  as  the 
vast  majority  of  the  grades  in  VT-23  were  at  3.0,  the  distribution  of  the  grades  is 
unimodal  to  the  extreme. 
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implicitly  state  that  the  difference  in  the  performance  evaluations  was  caused  by  the 
visual  simulator.  The  difference  may  also  be  attributed  to  individual  squadron  differences 
and  grading  trends.  Further  examination  of  the  performance  evaluations  is  required. 


D.  COMPARISON  OF  INDIVIDUAL  PERFORMANCE  AREAS 

Recall  that  the  CQ-llx  performance  evaluation  is  the  average  of  19  separate 
performance  areas.  Figure  7  presents  the  average  performance  grade  of  both  squadrons 
for  each  of  the  19  areas  evaluated  on  ATF  CQ-1  lx.  From  the  graph,  VT-19  appears  to 
have  significantly  lower  performance  scores  in  Pattern,  Start  Position,  Power  Control, 
Error  Detection/Correction  and  Glideslope  Control,  and  significantly  higher  performance 
scores  in  Waveoff  Technique,  Touch  and  Go/Bolter  Technique,  and  LSO  Response.  With 
some  subtle  differences,  the  squadrons  appear  to  grade  similarly  during  both  qualification 
periods.  (LSO's  in  VT-19  grade  somewhat  higher  during  July  in  the  areas  of  Waveoff 
Technique,  T&G/Bolter  Technique,  and  LSO  Response,  and  some  what  lower  in  the  areas 
of  Power  control  than  they  graded  in  May.) 


LSO  Pefonnance  Area  Averages 


Figure  7:  Squadron  Performance  Area  Averages,  Both  qualification  periods. 
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The  data  used  to  generate  Figure  7  come  from  an  ordinal  scale  of  values  "1",  "2", 
"3",  and  "4".  For  all  19  performance  areas,  in  both  squadrons,  the  median  value  is  "3", 
with  numerous  ties.  Quantifying  the  differences  identified  in  Figure  7  may  well  be  a 
futile  exercise,  but  two  nonparametric  tests  are  available  for  the  processing  of  such  data. 
The  Median  Test  tests  the  hypothesis  that  all  squadrons  have  the  same  median  against  the 
alternate  hypothesis  that  the  medians  are  different.  As  with  the  chi-square  test,  the 
Median  Test  requires  an  expected  value  of  one  in  each  cell  to  insure  accuracy  (Conover 
1980,  172).  Table  9  lists  the  results  of  the  Median  Test  for  each  of  the  19  performance 
areas.  In  most  cases,  data  equal  to  three  are  counted  as  equal  to  or  above  the  median. 
However  in  such  cases  where  there  are  few  or  no  data  points  below  the  median,  data 
values  equal  to  three  are  counted  as  below  the  median.  Grades  in  Attitude  Control,  Error 
Detection/Correction,  Waveoff  Technique,  Touch  and  Go/Bolter  Technique,  and  Response 
to  LSO  are  counted  using  the  latter  procedure. 

Results  of  the  Median  Tests  reject  the  equal  median  hypothesis  for  the  performance 
areas  of  Pattern,  Start  Portion,  Power  Control,  Glideslope  Control,  Touch  and  Go/Bolter 
Technique  and  Response  to  LSO.  These  six  performance  areas  correspond  to  six  of  the 
eight  areas  identified  in  Figure  7.  The  two  remaining  areas,  Error  Detection/Correction 
and  Waveoff  Technique,  did  not  fail  the  Median  Test.  It  is  interesting  to  note,  though  not 
surprising  by  any  means,  that  the  number  of  values  above  and  below  the  combined 
median  correspond  to  the  averages  illustrated  in  Figure  7.  Thus,  VT-19's  Pattern  average, 
much  lower  than  VT-23,  is  reflected  in  the  25  values  'below  the  median'  in  Table  9,  the 
highest  count  in  this  category.  Likewise,  VT-19  also  had  the  highest  average  in  LSO 
Response,  and  again,  with  a  count  of  nine,  has  the  highest  count  in  the  'above  the  median' 
category.  (Recall  the  rules  for  counting  "3"  as  above  or  below  the  median.) 
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The  Median  Test  is  a  robust  test,  especially  suited  for  data  with  expected  outliers 
or  error  .  Even  though  most  of  the  performance  grades  in  the  squadrons  are  "3",  whether 
grades  of  "1",  ”2",  and  "4"  can  be  considered  outliers  is  questionable.  Consequently,  a 
stronger  nonparametric  test  was  used  to  back  up  the  results  of  the  Median  Test.  The 
Kruskal-Wallis  Test  offers  a  nonparametric  alternative  to  a  one-way  analysis  of  variance 
and  tests  the  hypothesis  that  the  squadron's  grades  for  a  specific  performance  area  are 
identically  distributed  against  the  hypothesis  that  they  are  not  identically  distributed  and 
their  means  are  different  (Conover  1980, 230).  For  this  comparison,  the  performance  area 
grade  was  used  as  the  response  variable  and  squadron  was  used  as  the  factor.  The 
Kruskal-Wallis  Test  ranks  the  data  with  "1"  to  the  smallest  data  point,  "2"  to  the  next 
smallest,  and  so  on  to  the  largest  data  point.  The  ranks  are  then  used  to  compute  the  test 
statistic.  There  is  some  disagreement  surrounding  how  ties  should  be  handled  during  the 
ranking  process.  Conover  (1980,  232)  advocates  averaging  the  assigned  ranks,  so  if  the 
data  has  four  values  of  "3",  and  "3"  would  have  been  the  fourth  rank,  then  the  four  values 
are  initially  ranked  "4",  "5",  "6",  "7"  and  then  averaged  so  that  all  four  have  the  rank  5.5. 
Lehmann  (1975,  20)  argues  that  such  averaging  makes  the  test  invalid  in  data  sets  with 
large  numbers  of  ties.  He  proposes  an  adjusted  test  statistic  that  divides  the  original  test 
statistic  by  a  factor  that  considers  the  number  of  tied  data  points.  As  pointed  out  in 
previous  discussion,  the  performance  area  grades  have  numerous  ties,  most  notably  about 
the  grade  of  3.  Because  of  the  large  number  of  ties,  it  is  felt  that  Lehmann's  adjusted  test 
statistic  is  more  relevant  for  the  Kruskal-Wallis  Tests  that  follow. 

Table  10  presents  the  results  of  the  Kruskal-Wallis  Tests  used  to  examine  the 
individual  performance  areas.  While  the  adjusted  test  statistic  will  be  used  for  further 
analysis,  p-values  computed  by  both  test  statistics  are  listed  in  the  table.  The  squadrons 
had  different  performance  area  means  in  six  performance  areas:  Pattern,  Start  Position, 
Power  Control,  Glideslope  Control,  Touch  and  Go/Bolter  Technique  and  Response  to 
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Table  10:  Results  of  Kruskal-Wallis  Test  on  Performance  Area. 


LSO.  These  are  the  same  six  areas  identified  by  the  Median  Test.  It  is  interesting  to 
note  that  without  the  adjusted  test  statistic,  the  only  significant  performance  area  is 
Pattern,  which  is  also  the  performance  area  showing  the  largest  difference  between 
squadrons  in  Figure  7. 

It  is  clear  from  the  above  Kruskal- Wallis  tests  that  students  in  VT-19  had  lower 
grades  in  the  four  performance  areas  of  Pattern,  Start  Position,  Power  Control,  and 
Glideslope  Control  and  higher  performance  grades  in  Touch  and  Go/Bolter  Technique  and 
Response  to  LSO.  At  this  point  in  the  analysis,  it  would  seem  appropriate  to  look  at 
individual  LSO  difference,  to  determine  if  differences  within  a  squadron  contributed  to 
the  differences  between  the  squadrons.  Figures  8  and  9  illustrate  the  performance  area 
averages  of  individual  LSO's  within  VT-19  during  the  May  and  July  qualification  periods, 
respectively.  In  each  of  the  graphs,  if  the  all  the  LSO's  gave  an  average  of  "3"  in  a 
performance  area,  that  area  has  been  removed  from  the  graph. 

In  Figure  8,  it  can  be  seen  that  LSO  A  grades  high  when  the  other  LSO’s  grade  low 
and  has  a  much  lower  grade  in  Glideslope  control  than  any  other  LSO  in  the  squadron. 
At  first,  glance  VT-19  seems  to  have  less  variability  in  July  (Figure  9)  than  it  did  during 
May.  LSO  A  grades  more  similarly  to  the  other  LSO's,  albeit  somewhat  higher  in 
Attitude  Control  and  Line  Up.  LSO  C  has  a  much  higher  grade  in  Response  to  LSO  than 
the  other  LSO's.  To  examine  the  five  LSO's  in  VT-19  more  closely,  Kruskal  Wallis 
Tests,  similar  to  the  models  used  above,  examined  each  performance  area  pictured  in 
Figures  8  and  9.  The  Kruskal-Wallis  Tests  were  conducted  over  both  qualification 
periods,  using  the  LSO's  grades  from  both  periods.  Results  of  these  tests  are  listed  in 
Table  1 1 .  Italicized  rows  indicate  a  performance  area  that  tested  positively  for  significant 
difference  between  the  LSO's,  p-value  <  0.05.  Table  1 1  illustrates  the  variability  between 
the  LSO's  in  VT-19.  Seven  of  the  twelve  performance  areas  examined  have  significant 
differences  between  LSO's.  Of  the  six  performance  areas  noted  in  Table  10,  only  Pattern 
was  graded  consistently  by  the  LSO's. 
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VT-19  LSO  Pefonmance  Area  Averages,  May 


8:  VT-19  Performance  Area  Averages,  May. 


VT-19  LSO  Performance  Area  Averages,  July 


Figure  9:  VT-19  Performance  Area  Averages,  July. 
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Table  11:  Results  of  Kruskal-Wallis  Test  on  grades  in  selected  performance 
areas  by  LSO’s  in  VT-19,  both  qualification  periods. 
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Figures  10  and  11  illustrate  the  average  performance  grades  in  VT-23  during  May 
and  July  qualification  periods,  respectively.  Compared  to  VT-19,  VT-23  displays  much 
less  variability.  In  May  (Figure  10),  no  LSO  stands  out  as  an  individual,  although  LSO 
J  has  higher  averages  in  the  areas  of  speed  control  and  attitude.  At  first  glance.  Figure 
11  looks  disjointed  and  appears  to  suggest  great  variability  between  VT-23's  LSO's 
during  July.  However,  closer  examination  reveals  that  except  for  Line  Up  and  Glideslope 
Control,  only  one  LSO  grades  differently  from  the  other  five  LSO's. 

The  lack  of  variability  revealed  in  Figures  10  and  11  is  reflected  in  the  results  of 
the  Kruskal-Wallis  Tests  used  to  examine  differences  between  the  LSO's  in  VT-23  listed 
in  Table  12.  Only  Attitude  Control  and  Response  to  LSO  display  significant  differences 
between  the  LSO's. 
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Figure  11:  VT-23  Performance  Area  Averages,  July. 
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Table  12:  Results  of  Kruskal-Wallis  Test  on  grades  in  selected  performance 
areas  by  LSO's  in  VT-23,  both  qualification  periods. 
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Thus  comparison  of  LSO's  within  a  squadron  indicate  considerable  variability  in 
VT-19  and  somewhat  less  difference  in  VT-23.  Of  the  six  areas  identified  in  Figure  7 
and  confirmed  in  Table  9  at  the  beginning  of  this  chapter,  only  grades  in  Pattern  can  be 
considered  unaffected  by  difference  between  the  LSO's  within  a  squadron.  The 
differences  in  Pattern  between  the  squadrons,  appear  to  be  just  that,  differences  between 
the  squadrons. 


IV.  DISCUSSION 


Comparison  of  students  in  VT-23  to  VT-19  revealed  no  significant  differences  in 
disqualification  rates  or  LSO  trend  analysis  scores.  However,  students  in  VT-23  had 
higher  performance  evaluations  than  students  in  VT-19.  Moreover,  the  differences  in 
performance  evaluations  appear  to  emphasize  differences  primarily  in  the  area  of  Pattern. 
As  Figure  7  and  subsequent  nonparametric  tests  indicate,  VT-23  had  higher  grades  in 
Pattern  than  students  in  VT-19.  Isolating  the  cause  of  VT-19's  higher  Pattern  grades, 
however,  is  a  difficult  task. 

As  has  been  discussed  in  previous  chapters,  LSO  grades  are  somewhat  subjective. 
Westra  argues  in  his  study  that  LSO  grades  are  too  subjective  to  form  any  concrete 
results.  Westra  points  out  that  LSO's  sometimes  use  grades  as  a  motivational  factor  to 
encourage  students  to  perform  better  in  subsequent  phases  of  their  training,  thus 
decreasing  objectivity  and  validity  (Westra  et  al.  1985,  41).  However,  Westra's  study 
concentrated  on  the  Field  Carrier  Landing  Practice  (FCLP)  portion  of  the  carrier 
qualification  process  and  he  studied  only  the  LSO  Trend  Analysis  Scores,  not  the  CQ 
performance  evaluations.  (FCLP's  are  recorded  on  ATF’s  CQ-lx  through  CQ-lOx.)  When 
he  considered  the  actual  carrier  qualification,  the  LSO  grades  for  students  who  used  the 
simulator  were  somewhat  lower  than  the  grades  for  students  who  did  not  use  the 
simulator,  but  the  difference  was  not  statistically  significant.  In  this  study,  students  with 
exposure  to  the  simulator  had  somewhat  higher  LSO  trend  analysis  scores  than  students 
who  did  not  have  exposure,  but  the  difference  was  statistically  insignificant  as  well. 

Examination  of  the  CQ-llx  Performance  Evaluation  Averages  revealed  significant 
differences,  albeit  across  a  small  range,  between  students  in  VT-23  and  VT-19.  Further 
analysis  identified  the  most  significant  difference  between  the  squadrons  appeared  in  the 
grades  for  Pattern.  Nonparametric  tests  also  identified  significant  differences  in  Start 
Position,  Power  Control,  Glideslope  Control,  Touch  and  Go/Bolter  Technique,  and 
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Response  to  LSO.  However,  when  these  five  performance  areas  are  examined  within 
individual  squadrons,  significant  differences  exist  between  the  LSO's.  Pattern  is  the  only 
performance  area  without  significant  difference  between  LSO's  within  the  squadrons. 

The  question  still  remains  as  to  the  cause  of  the  difference  in  the  Pattern  Grades. 
Because  each  squadron  used  different  LSO's,  it  is  impossible  to  rule  out  grading 
differences  between  the  squadrons.  However,  as  the  carrier  qualification  is  the  last 
requirement  for  the  carrier  qualification  stage  of  their  training,  there  is  no  longer  the 
motivational  requirement  for  grades  suggested  by  Westra,  therefore  some  of  the 
subjectivity  may  be  ignored.  Indeed,  previous  research  supports  the  claim  that  part  of  the 
difference  in  Pattern  grades  may  be  attributed  to  exposure  to  the  visual  simulator.  Both 
Lintem  and  Westra  found  that  use  of  a  visual  simulator  improved  their  subject's  spatial 
awareness.  Lintem  used  a  visual  simulator  to  train  beginning  civil  flight  students  before 
the  landing  phase  of  their  instruction.  He  suggests  that  approximately  2  hours  of  landing 
training  in  a  visual  simulator  can  reduce  the  number  of  pre-solo  landings  in  an  aircraft 
(Lintem  et  al.  1990,  324).  While  Lintem  admits  to  the  possibility  of  instmctor  bias  in 
determining  the  correct  number  of  pre-solo  landings,  he  attempted  to  counter  such  bias 
by  not  telling  the  instmctors  that  the  number  of  pre-solo  landings  would  be  the  measure 
of  effectiveness  for  the  study.  Westra  found  a  significant  advantage  in  approach  lineup 
control  for  students  who  had  trained  on  a  visual  simulator  (Westra  et  al.  1985, 49).  These 
findings  were  supported  using  equipment  observations  as  opposed  to  human  evaluation. 
For  his  study  Westra  used  a  special  device  that  recorded  the  pilot's  approach  to  the  air 
field  and  recorded  deviations  from  the  correct  line  up  approach. 

A.  THE  IMPORTANCE  OF  PATTERN 

At  sea.  Pattern  is  the  element  that  drives  a  pilot's  entire  approach  to  the  carrier.  If 
the  Pattern  is  correct,  then  the  Line  Up,  Power  Control,  and  Glideslope  Control  should 
follow  closely  behind.  In  the  Flight  Training  Instmction  Manual  for  the  T-2C,  it  states: 
"The  landing  pattern  is  the  key  to  successful  hall  flying.  This  cannot  be  emphasized 
enough.  Your  primary  goal  during  the  early  stages  of  FAM  [familiarization  stage]  is  to 
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fly  a  consistent,  precise,  pattern  resulting  in  consistent  starts.  When  you  are  able  to  do 
so,  then  you  can  think  about  flying  the  ball  (CNATRA  P-650,  1993,  3-10  -  3-11)." 
Students  begin  practicing  the  proper  landing  problem  in  the  second  stage  of  their 
instruction.  (Recall  that  the  Carrier  Qualification  Stage  occurs  during  the  ninth  and  final 
stage  of  Intermediate  flight  instruction.)  Figure  12  illustrates  the  standard  approach 
pattern  flown  by  students  throughout  their  instruction.  Although  students  are  taught  to 
make  most  of  their  turns  using  the  cockpit  instruments,  they  are  also  encouraged  to  use 
visual  landmarks  on  the  ground  while  learning  the  approach.  These  visual  aids  are  not 
available  to  the  student  during  a  carrier  approach,  and  while  the  student  is  constantly 
reminded  of  this  fact,  it  must  certainly  be  disorientating  during  the  first  few  approaches 
to  the  ship.  One  advantage  of  the  T-45  visual  simulator  may  be  an  awareness  of  this 
spatial  disorientation.  In  the  simulator,  the  student  is  given  an  opportunity  to  observe  the 
relatively  small  deck  of  a  carrier  against  a  massive,  featureless  ocean,  and  has  the 
opportunity  to  witness  a  simulated  instrument  approach  to  the  carrier.  If  two  hours  of 
visual  simulator  practice  was  enough  to  reduce  pre-solo  landing  flights,  then  possibly  two 
hours  of  simulator  exposure  is  enough  to  enhance  a  student's  awareness  of  the  importance 
of  instruments  in  flying  the  proper  pattern.  Such  awareness  might  encourage  students  to 
place  more  emphasis  on  their  pattern,  thus  resulting  in  better  performance. 

B.  AREAS  FOR  FURTHER  STUDY 

At  the  completion  of  this  thesis,  approximately  121  students  will  have  completed 
some  portion  of  the  T-45  training  program,  either  by  flying  T-45A  aircraft  for  their  entire 
training  or  by  transitioning  to  the  T-45A  for  Advanced  training  after  flying  the  T-2C  for 
Intermediate  training.  As  more  T-45  aircraft  are  built,  the  T-2C  and  A-4  training 
squadrons  in  Meridian,  Mississippi  will  eventually  be  phased  out  and  all  Naval  jet  flight 
training  will  be  conducted  in  the  T-45  aircraft."*  Until  this  phase  out  is 


"•with  the  exception  of  those  pilots  designated  to  fly  the  E-2/C-2  Hawkeye  Electric 
Reconnaissance  Aircraft. 
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Figure  12:  Standard  Pattern  Taught  for  Carrier  Landings.  (Flight  Training 
Instruction:  Intermediate  Strike  and  Advanced  E-2/C-2,  September  1993, 
p.  3-13). 
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complete,  however,  the  opportunity  exists  to  compare  the  T-45  squadron  in  Kingsville  to 
the  A-4  squadron  in  Meridian.  The  T-45  training  program  has  the  ability  to  gather  and 
record  detailed  data  on  students  in  the  program.  With  prior  planning,  a  similar  recording 
system  could  be  established  for  the  A-4  program.  Granted,  the  use  of  different  aircraft 
introduces  variance  to  the  comparison,  but  such  variance  should  not  nullify  all 
observations.  In  the  long  term,  T-45  students  should  be,  and  undoubtedly  are  being, 
tracked  as  they  progress  beyond  the  training  squadron.  The  integration  of  the  T-45 
students  into  the  FRS  and  Fleet  Squadrons  should  be  compared  to  the  integration  of  the 
A-4  students.  Does  the  training  program  help  the  new  aviators  transition  to  their  new 
platforms? 

The  emphasis  of  these  comparisons  must  be  on  the  training  program.  The  T-45 
training  program  is  such  a  radically  different  approach  to  training,  incorporating 
traditional  lectures  with  high  tech  computer  based  training  and  visual  simulators,  it  would 
seem  impossible  to  isolate  the  effect  of  the  visual  simulator  from  the  other  training  aides. 
However,  as  the  simulator  plays  such  an  important  role  in  the  program,  its  effects  will  not 
be  ignored. 
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V.  CONCLUSION 


This  thesis  compared  two  T-2C  training  squadrons  to  see  if  exposure  to  the  T-45 
visual  simulator  had  any  effects  on  student  naval  aviators  during  carrier  qualifications. 
Student  Naval  Aviators  in  VT-23  received  exposure  to  the  T-45  simulator.  SNA’s  in 
VT-19  did  not  train  with  the  simulator.  Both  squadrons  completed  their  final  carrier 
qualifications  concurrently,  flying  on  the  same  aircraft  carrier.  Four  measures  of 
effectiveness  were  examined:  Qualification  Rates,  LSO  Trend  Analysis  Scores,  Average 
CQ-llX  Performance  Evaluations,  and  grades  in  the  Individual  Performance  Areas 
considered  by  CQ-1 IX.  Comparison  of  the  squadrons  found  no  difference  in  qualification 
rates  or  LSO  Trend  Analysis  Scores  and  significant  differences  in  Average  Performance 
Evaluations  and  in  some  Individual  Performance  Areas,  most  notably  Pattern,  Start 
Position,  Power  Control,  Glideslope  Control,  Touch  and  Go/Bolter  Technique,  and 
Response  to  LSO.  As  this  study  compared  grades  based  on  human  evaluation,  some 
caution  should  be  used  in  interpreting  the  results.  However,  the  T-45  visual  simulator 
does  appear  to  have  improved  the  ability  of  students  to  fly  a  proper  pattern  as  they 
approach  the  aircraft  carrier. 
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APPENDIX  A.  CARRIER  QUALIFICATION  ATF  PERFORMANCE  AREAS 


Perfonnance  Area 

CQ-1 

CQ-2X 

ATF 

CQ-3,4,5,6,7,8,9 

CQ-lOx 

CQ-llx 

Headwork 

X 

X 

X 

X 

X 

Procedures 

X 

X 

X 

X 

X 

Basic  Airwork 

X 

X 

X 

X 

X 

Pattern 

X 

X 

X 

X 

X 

Start  Position 

X 

X 

X 

X 

X 

Speed  Control 

X 

X 

X 

X 

X 

Attitude  Control 

X 

X 

X 

X 

X 

Power  Control 

X 

X 

X 

X 

X 

Line  Up 

X 

X 

X 

X 

X 

Glideslope  Control 

X 

X 

X 

X 

X 

Error  Detection/Correctior 

X 

X 

X 

X 

X 

WaveoflF  Technique 

X 

X 

X 

X 

X 

Course  Rules 

X 

X 

Pattern  Entry/Break 

X 

X 

Response  to  LSO 

X 

X 

X 

X 

Progress 

X 

X 

T&G/Bolter  Technique 

X 

Fomiation/Pattem  Entry 

X 

Radio  Procedures 

X 

Fuel  Management 

X 

Deck  Procedures 

X 

Catsuit  Procedures 

X 
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APPENDIX  B.  LSO  MARKS  AND  GRADES 


Symbol 

Definition 

Grade 

OK 

Perfect  Pass 

5.0 

OK 

Reasonable  deviations  with  good  corrections 

4.0 

(OK) 

Fair,  reasonable  deviations 

3.0 

B 

Bolter 

2.5 

- 

Below  average  but  safe  pass 

2.0 

PWO 

Power  waveoff 

2.0 

owo 

Own  waveoff 

2.0 

wo 

Waveoff 

1.0 

c 

Cut.  Unsafe,  gross  deviations  inside 

waveoff  window 

0.0 
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APPENDIX  C.  SUMMARY  OF  WEATHER  OBSERVATIONS 


USS  AMERICA  (CV  66),  May  Qualification  Period 


Date  Time 

Z 

Location 

Wind  Speed 
(kts) 

Visibility 

(KM) 

Temperature 

(Celsius) 

14-May  0001 

N17.1  W065.3 

080 

22 

10 

28.3 

0600 

N17.2W065.8 

100 

18 

10 

26.1 

1200 

N17.5W066.3 

110 

17 

10 

26.6 

1800 

N17.1  W065,6 

110 

14 

20 

27.8 

15-May  0001 

M17.2  W065.4 

100 

15 

10 

27.8 

0600 

N18.5  W065.1 

110 

18 

10 

26.1 

1200 

Nfl8.9  W065.4 

110 

14 

20 

26.7 

1800 

M19.4W065.6 

120 

9 

20 

26.7 

16-May  0001 

N2L2W066.6 

110 

8 

20 

25.6 

0600 

N23.2  W066.5 

180 

9 

10 

25.0 

1200 

M25.1  W068.5 

030 

8 

20 

24.5 

1800 

N27.0  W069.5 

010 

10 

20 

25.0 

17-May  0001 

1S[28,8  W070.9 

020 

6 

20 

21.9 

0600 

N30.7W071.5 

230 

9 

10 

22.2 

1200 

vf32.6  W072.4 

250 

12 

20 

21.7 

1800 

M33.9  W073.5 

200 

13 

20 

22.2 

18-May  0600 

M36.6  W075.2 

080 

10 

10 

17.2 

1200 

130 

2 

Fog 

16.1 

Date 

Time 

Z 

Total  Clouds 
(oktas) 

Lower  Clouds 
(oktas) 

Cloud  Height 
(ft) 

Ship  Speed 
(kts) 

14- May 

7 

2 

1000-1999 

1 

W 

1-5 

0600 

4 

2 

3500-4999 

1 

W 

16-20 

1200 

2 

2 

2000-3499 

1 

NE 

6-10 

1800 

4 

2 

2000-3499 

1 

SE 

6-10 

15 -May 

0001 

6 

2 

2000-3499 

1 

S 

1-5 

0600 

2 

2 

>8000 

1 

N 

16-20 

1200 

6 

1 

1000-1999 

0 

0 

1800 

6 

1 

>8000 

HH 

NW 

11-15 

16-May 

0001 

8 

8 

>8000 

NW 

21-25 

0600 

3 

3 

>8000 

1/2 

NW 

21-25 

1200 

6 

1 

2000-3499 

1/2 

NW 

16-20 

1800 

6 

1 

2000-3499 

1/2 

NW 

21-25 

17-May 

0001 

4 

2 

2000-3499 

1/2 

NW 

21-25 

0600 

1 

1 

2000-3499 

1/2 

NW 

16-20 

1200 

1 

1 

2000-3499 

1/2 

NW 

16-20 

1800 

2 

0 

>8000 

1/2 

NW 

16-20 

18-May 

0600 

1 

1 

2000-3499 

1 

NW 

16-20 

1200 

8 

4 

300-599 

1/2 

E 

6-10 
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USS  JOHN  F.  KENNEDY  (CV  67),  July  Qualification  Period 


Date  Time 

Z 

Location 

Wind  Speed 
(kts) 

Visibility 

(KM) 

Temperature 

(Celsius) 

23-Jul  0001 

S130.9  W080.^ 

220 

20 

20 

28.6 

0600 

Sf31.0W079.g 

240 

12 

10 

27.2 

1200 

^130.9W079.4 

270 

13 

10 

27.8 

1800 

M31.2  W080.4 

240 

5 

20 

28,9 

24-Jul  0001 

m7W080.S 

220 

15 

20 

26.7 

0600 

vf30.2W080J 

210 

12 

10 

26.1 

1200 

'430.1  W079.S 

260 

8 

20 

27.8 

1800 

Si30.6W079.8 

240 

5 

20 

30.6 

25-Jul  0001 

M30.2  W080.C 

170 

16 

20 

28.9 

0600 

M29.9  W080.C 

210 

8 

10 

26.7 

1200 

^130.0  W079.8 

250 

12 

20 

27.8 

1800 

S30.5  W080.C 

230 

10 

20 

29.4 

26-Jul  0001 

M30.3  W080.5 

170 

10 

20 

26.7 

0600 

S[30.6W080.C 

250 

10 

10 

26.7 

1200 

M30.6  W079.7 

280 

20 

20 

27.2 

1800 

M30.9W080.1 

250 

6 

10 

26.7 

27-Jui  0001 

S[30.4  W080.1 

230 

6 

10 

27.8 

0600 

M30.7  W079.7 

340 

8 

10 

25.6 

1200 

SI30.8W079.8 

330 

13 

20 

27.8 

1800 

M31.0W080.C 

350 

9 

20 

28.9 

Date 

Time 

Z 

Total  Clouds 
(oktas) 

Lower  Clouds 
(oktas) 

Cloud  Height 
(ft) 

Wave  Height 
(m) 

Ship  Direction 

Ship  Speed 
(kts) 

23-Jui 

0001 

4 

2 

2000-3499 

1/2 

16-20 

0600 

3 

0 

>8000 

1/2 

11-15 

1200 

4 

2 

2000-3499 

1/2 

11-15 

1800 

4 

0 

>8000 

1/2 

16-20 

24-Jul 

0001 

6 

5 

>8000 

1/2 

S 

11-15 

0600 

4 

2 

2000-3499 

1 

SE 

11-15 

1200 

6 

2 

2000-3499 

1/2 

W 

11-15 

1800 

6 

3 

2000-3499 

1/2 

E 

11-15 

25-Jul 

0001 

3 

1 

2000-3499 

1/2 

SE 

16-20 

0600 

2 

2 

2000-3499 

i/2 

s 

6-10 

1200 

5 

1 

2000-3499 

1/2 

W 

6-10 

1800 

6 

2 

2000-3499 

1/2 

N 

11-15 

26-Jul 

0001 

4 

2 

2000-3499 

1/2 

SW 

11-15 

0600 

1 

0 

>8000 

1/2 

NE 

11-15 

1200 

6 

5 

1000-1999 

1/2 

S 

6-10 

1800 

8 

6 

2000-3499 

1/2 

S 

6-10 

27-Jul 

0001 

4 

1 

>8000 

1/2 

NW 

6-10 

0600 

6 

3 

2000-3499 

1/2 

SE 

1-5 

1200 

5 

1 

2000-3499 

1/2 

N 

11-15 

1800 

3 

2 

2000-3499 

1/2 

SE 

ie-20 

APPENDIX  D.  CONTINGENCY  TABLES  FOR  COMPARISON  OF 
SQUADRON  DISQUALIFICATION  RATES 


Table  Dl:  Contingency  Table  for  May 
Qualification  Period,  Both  Squadrons. 


Squadron 


VT-19 


VT-23 


Total 


Student 

Qualification  I  Disqualification!  Total 


4 

3.54 


3 

3.46 


7 

7 


m 

[iggi 
SI 


I 


Chi-Square  0.131  withD.F. 

Cell  Contents:  Count 

Expected  Frequency 


Table  D2:  Contingency  Table  for  July 
Qualification  Period,  Both  Squadrons. 


Student 


Squadron 

Qualification 

Disqualification 

Total 

VT-19 

36 

2 

38 

35.29 

2.71 

38 

VT-23 

29 

3 

32 

29.71 

2.29 

32 

Total 

65 

5 

mmm 

65 

5 

■B 

Chi-Square 

0.443 

with  D.F. 

1 

Cell  Contents:  Count 

Expected  Frequency 


Table  D3:  Contingency  Table  for  VT-19, 
Both  Qualification  Periods. 


Month 


May 


Student 

Qualification  |  Disqualification  |  T otal 


40 

4 

44 

40.78 

3.22 

44 

36 

2 

38 

35.22 

2.78 

38 

76 

6 

82 

76 

6 

82 

Chi-Square  0.441  with  D.F. 

Cell  Contents:  Count 

Expected  Frequency 


Table  D4:  Contingency  Table  for  VT-23, 
Both  Qualification  Periods. 


Month 


May 


Qualification 

Student 

Disqualification 

40 

3 

39.56 

3.44 

29 

3 

29.44 

2.56 

69 

6 

69 

6 

Chi-Square  0. 143  with  D.F. 

Cell  Contents:  Count 

Expected  Frequency 


Table  D5:  Contingency  Table  for  Both  Squadrons,  Both  Qualification 
Periods. 


May  Students 

Jufy  Students 

Squadron 

Qualification 

Disqualification 

Qualification 

Disqualification 

Total 

vr-19 

4 

36 

2 

82 

41.78 

3.66 

33.95 

2.61 

82 

vr-23 

40 

3 

29 

3 

75 

38.22 

3.34 

31.05 

2.39 

75 

Total 

80 

7 

65 

5 

75 

80 

7 

65 

5 

75 

Chi-Square 

0.786 

with  D.F. 

3 

Cell  Contents:  Count 

Bpected  Frequency 
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APPENDIX  E.  ANALYSIS  OF  VARIANCE  MODEL  USED  FOR  LSO  TREND 
ANALYSIS  AND  CQ-llX  PERFORMANCE  EVALUATION  COMPARISONS 


The  same  model  is  assumed  for  both  analyses.  Recall  that  data  has  been  observed 
in  each  of  two  months  (May,  July)  for  each  of  two  squadrons  (VT-19,  VT-23).  Each 
squadron  has  its  own  set  of  LSO's,  assumed  to  be  selected  at  random  from  a  pool  of  such 
people;  some  LSO's  participated  in  both  qualification  periods,  some  did  not.  The  number 
of  students  graded  by  a  given  LSO  also  varies.  Letting  represent  the  grade  given 
by  LSO  k  in  squadron  i  during  month  j.  The  model  states  that; 


y  ijki 

where  a, 
{ocP).j 

Yci/lk 


^Ijkl 

for  / 

j 

k 

I 


=  ^1+  a,  ^  I3j  +  {aP)y  + 

=  effect  due  to  month  /, 

=  effect  due  to  squadron  j, 

=  effect  of  the  interaction  of  month  /  with  squadron  y, 
=  effect  of  LSO  k  (nested  within  month/squadron, 
combinations),  and 

=  effect  due  to  error  from  grade  /  by  LSO  A:  in 
squadron  /,  during  month  j 

=  1,2, 

=  1,2, 

=  l.../iy,  where  is  the  number  of  LSO's  in  squadron  / 
during  month  j,  and 

=  \  where  n^j  is  the  number  of  grades  given  by 
LSO  k  in  squadron  /  during  month  j. 


It  is  assumed  that  the  oCj,  Pj ,  {ap)y  values  are  unknown  constants,  or  fixed  effects, 
while  Xi(y)  are  independent,  normal  (0,  ct^^),  random  effects  and  are  independent, 
normal  (0,  <y^)  and  mutually  independent  of  the  random  effects. 
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As  the  number  of  LSO’s  varies  by  squadron  and  month  and  the  number  of  students 
graded  by  a  given  LSO  also  varies,  the  data  available  is  unbalanced.  The  unbalanced 
characteristic  of  the  data  causes  the  estimates  of  the  fixed  effects  to  be  nonorthogonal  or 
correlated.  As  a  result,  the  effects  of  month,  squadron,  and  their  interaction,  as  measured 
by  the  reduction  in  the  residual  sum  of  squares  given  by  their  inclusion  in  the  model, 
depends  on  the  other  terms  already  in  the  model.  In  other  words,  when  month  enters  into 
the  model,  its  effect  will  depend  on  the  other  terms  entered  into  the  model  before  month. 
A  different  fixed  effect  may  result  if  month  had  entered  the  model  first.  To  compensate 
for  the  correlation  in  the  data,  the  significance  of  these  fixed  effects  is  judged  by  their 
"marginal"  reduction  in  the  residual  sum  of  squares,  the  reduction  caused  if  all  other 
terms  are  also  in  the  model.  For  example,  the  adjusted  sum  of  squares  for  month  is  the 
difference  between  the  residual  sum  of  squares  when  the  the  model  considers  all  variable 
except  month  and  the  residual  sum  of  squares  when  the  model  considers  all  variables 
including  month. 
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